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(57) Abstract 

This invention discloses apparatus for 
detecting the emotional status of an individ- 
ual, the apparatus including a voice analyzer 
(760) operative to input (710. 720. 730) a 
speech specimen generated by the individ- 
ual and to derive therefrom intonation in- 
formation, and an emotion reporter opera- 
tive to generate an output indication of the 
individual's emotional status based on the 
intonation information (735). Methods for 
multi-dimensional lie detection and for de- 
tecting emotional status are also disclosed. 
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APPARATUS AND METHODS FOR DETECTING EMOTIONS 

FIELD OF THE INVENTION 

The present invention relates to apparatus and methods for monitoring 
emotional states. 

BACKGROUND OF THE INVENTION 

Published PCT Application WO 97/01984 (PCT/TL96/00027) describes a 
method for effecting biofeedback regulation of at least one physiological variable 
characteristic of a subject's emotional state, including the steps of monitoring at least one 
speech parameter characteristic of the subject's emotional state so as to produce an 
indication signal, and using the indication signal to provide the subject with an indication 
of the at least one physiological variable. A system permits the method to be carried out 
in standalone mode or via the telephone line in which case the indication signal may be 
derived at a location remote from the subject. Information relating to the subject's 
emotional state can be conveyed vocally to a remote party or textually through the 
Internet, and then processed as required. 

Published European Patent Application No. 94850185.3 (Publication No. 
306 664 537 A2) describes a method and arrangement for determining stresses in a 
spoken sequence. From a sequence recognized in the spoken speech, a model of the 
speech is created. By comparing the spoken sequence with the modeled speech, a 
difference between them is obtained. 

U.S. Patent 1,384,721 describes a method and apparatus for physiological 
response analysis. 
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U.S. Patent 3,855,416 to Fuller describes a method and apparatus for 
phonation analysis leading to valid truth/lie decisions by fundamental speech-energy 
weighted vibratto component assessment. 

U.S. Patent 3,855,417 to Fuller describes a method and apparatus for 
phonation analysis lending to valid truth/lie decisions by spectral energy region 
comparison. 

U.S. Patent 3,855,418 to Fuller describes a method and apparatus for 
phonation analysis lending to valid truth/lie decisions by vibratto component assessment. 

The disclosures of all publications mentioned in the specification and of 
the publications cited therein are hereby incorporated by reference. 

SUMMARY OF THE INVENTION 

The present invention seeks to provide improved apparatus and methods 
for monitoring emotional states. 

There is thus provided, in accordance with a preferred embodiment of the 
present invention, apparatus for detecting emotional status of an individual, the apparatus 
including a voice analyzer operative to input a speech specimen generated by the 
individual and to derive therefrom intonation information, and an emotion reporter 
operative to generate an output indication of the individual's emotional status based on 
the intonation information. 

Further in accordance with a preferred embodiment of the present 
invention, the speech specimen is provided over the telephone to the voice analyzer. 

Still further in accordance with a preferred embodiment of the present 
invention, the report on the individual's emotional status includes a lie detection report 
based on the individual's emotional status. 

Further in accordance with a preferred embodiment of the present 
invention, the intonation information includes multidimensional intonation information. 

Still further in accordance with a preferred embodiment of the present 
invention, the multidimensional information includes at least 3 -dimensional information. 
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Further in accordance with a preferred embodiment of the present 
invention, the multidimensional information includes at least 4-dimensional information. 

Still further in accordance with a preferred embodiment of the present 
invention, the intonation information includes information pertaining to thorns. 

Further in accordance with a preferred embodiment of the present 
invention, the information pertaining to thorns includes the number of thorns in a 
predetermined time period. 

Further in accordance with a preferred embodiment of the present 
invention, the information pertaining to thorns includes the distribution of thorns over 
time. 

Additionally in accordance with a preferred embodiment of the present 
invention, the intonation information includes information pertaining to plateaus. 

Further in accordance with a preferred embodiment of the present 
invention, the information pertaining to plateaus includes the number of plateaus in a 
predetermined time period. 

Still further in accordance with a preferred embodiment of the present 
invention, the information pertaining to plateaus includes information pertaining to length 
of plateaus. 

Additionally in accordance with a preferred embodiment of the present 
invention, the information pertaining to length of plateaus includes an average plateau 
length for a predetermined time period. 

Still further in accordance with a preferred embodiment of the present 
invention, the information pertaining to length of plateaus includes the standard error of 
plateau length for a predetermined time period. 

Also provided, in accordance with another preferred embodiment of the 
present invention, is a lie detection system including a multidimensional voice analyzer 
operative to input a speech specimen generated by an individual and to quantify a 
plurality of characteristics of the speech specimen, and a credibility evaluator reporter 
operative to generate an output indication of the individual's credibility, including 
detection of lies, based on the plurality of quantified characteristics. 
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Additionally provided, in accordance with another preferred embodiment 
of the present invention, is a detection method including receiving a speech specimen 
generated by an individual and quantifying a plurality of characteristics of the speech 
specimen, and generating an output indication of the individual's credibility, including 
detection of lies, based on the plurality of quantified characteristics. 

Further in accordance with a preferred embodiment of the present 
invention, the speech specimen includes a main speech wave having a period and wherein 
the voice analyzer is operative to analyze the speech specimen in order to determine rate 
of occurrence of plateaus, each plateau indicating that a local relatively low-frequency 
wave is superimposed onto the main speech wave, and the emotion reporter is operative 
to provide a suitable output indication based on the rate of occurrence of plateaus. For 
example, the emotion reporter may provide a suitable output indication when the rate of 
occurrence of plateaus is found to change. 

Similarly, each thorn indicates that a local relatively high-frequency wave 
is superimposed onto the main speech wave. A particular advantage of analyzing 
plateaus and thorns as shown and described herein is that substantially all frequencies of 
the speech wave may be analyzed. 

Also provided, in accordance with another preferred embodiment of the 
present invention, is a method for detecting emotional status and including establishing a 
multidimensional characteristic range characterizing an individual's range of emotion 
when at rest by monitoring the individual for a plurality of emotion-related parameters, 
over a first period during which the individual is in an emotionally neutral state, and 
defining the multi-dimensional characteristic range as a function of the range of the 
plurality of emotion-related parameters during the first period, and monitoring the 
individual for the plurality of emotion-related parameters, over a second period during 
which it is desired to detect the individual's emotional status, thereby to obtain a 
measurement of the plurality of emotion-related parameters, and adjusting the 
measurement to take into account the range. 

Also provided, in accordance with another preferred embodiment of the 
present invention, is a method for detecting emotional status of an individual, the method 
including receiving a speech specimen generated by the individual and deriving therefrom 
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intonation information, and generating an output indication of the individual's emotional 
status based on the intonation information. 



BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated from the 
following detailed description, taken in conjunction with the drawings in which: 

Fig. 1A is a pictorial illustration of a system for on-line monitoring of a 
speaker's emotional state, 

Fig. IB is a simplified flowchart illustration of a preferred method for on- 
line monitoring of a speaker's emotional state, 

Fig. 2 is a graphic illustration of a voice segment including a number of 

thorns, 

Fig. 3 is a graphic illustration of a voice segment including a number of 

plateaus, 

Fig. 4 is a simplified flowchart illustration of a preferred method for 
performing step 40 of Fig. IB, 

Fig. 5 is a simplified flowchart illustration of a preferred method for 
implementing the truth/neutral emotion profile building step of Fig. IB, 

Fig. 6 is a amplified flowchart illustration of a preferred method for 
performing step 90 of Fig. IB on a particular segment, 

Fig. 7 is a simplified flowchart illustration of a preferred method for 
performing step 100 of fig. IB, 

Fig. 8 is a amplified flowchart illustration of a preferred method for 
performing step 105 of Fig. IB, 

Fig. 9 is a pictorial illustration of a screen display depicting the form, in 
design mode, just before starting the application of Appendix A, 

Fig. 10 is a pictorial illustration of a screen display depicting the form, in 
the run mode of the system of Appendix A, during calibration to a particular subject, 
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Fig. 1 1 is a pictorial illustration of a screen display depicting the form, in 
the run mode of the system of Appendix A, during testing of a subject, and 

Fig. 12 is a simplified block diagram illustration of a preferred system for 
performing the method of Fig. IB. 

Attached herewith is the following appendix which aids in the 
understanding and appreciation of one preferred embodiment of the invention shown and 
described herein: 

Appendix A is a computer listing of a preferred software implementation 
of a preferred embodiment of the invention shown and described herein. 



DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

A portion of the disclosure of this patent document contains material 
which is subject to copyright protection. The copyright owner has no objection to the 
facsimile reproduction by anyone of the patent document or the patent disclosure, as it 
appears in the Patent and Trademark Office patent file or records, but otherwise reserves 
all copyright rights whatsoever. 

Fig. 1A is a pictorial illustration of a system for on-line monitoring of a 
speaker's emotional state. As shown, a speech input arriving over a telephone line, in the 
illustrated embodiment, is received by the system. The system analyzes the speech input 
in order to obtain an indication of the speaker's emotional state which indication is 
preferably provided to the user in real time, e.g. on the display screen as shown. 

Fig. IB is a simplified flowchart illustration of a preferred method for on- 
line monitoring of a speaker's emotional state. The method of Fig. IB preferably includes 
the following steps: 

Initialization step 10: Constants are defined such as the threshold values 
of various parameters, defining ranges which are considered to be indicative of various 
emotions, as described in detail below. 

Step 20: Record a voice, periodically or on demand. For example, 
segments of 0.5 seconds of voice may be recorded continuously, i.e. each 0.5 sees. 
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Alternatively, segments of any other suitable length may be considered which may or 
may not overlap. For example, adjacent segments may overlap almost entirely, except for 
one or a few samples. 

Digitize the voice recording. 

Additionally or alternatively overlapping segments of the recording may 

be sampled. 

Step 30: Analyze voice segment in order to mark the crucial portion of 
the voice segment, i.e. the portion of the voice segment which is thought to actually 
contain voice information as opposed to background noise. A suitable criterion for voice 
information detection is amplitude, e.g. the first instance of amplitude which exceeds a 
threshold is deemed the beginning of the voice information and the end of the voice 
information is deemed the point after which no threshold-exceeding sound is found for a 
predetermined duration. 

Preferably, the samples in the crucial portion are normalized e.g. by 
amplifying the samples to take advantage of the entire range of amplitude which can be 
accommodated in memory, e.g. +/- 127 amplitude units if 8-bit memory is employed. 

Step 40: Count thorns and plateaus in the crucial portion. Compute length 
of each identified plateau, and compute average plateau length for the crucial portion and 
standard error for the plateau length. 

A "thorn* is a notch-shaped feature. For example, the term "thorn" may 

be defined as: 

a. a sequence of 3 adjacent samples in which the first and third samples are 
both higher than the middle sample, or 

b. a sequence of 3 adjacent samples in which the first and third samples are 
both lower than the middle sample. 

Preferably, a thorn is declared even if the first and third samples differ 
only very slightly from the middle sample, i.e. there is preferably no minimum threshold 
value for the difference between samples. However, there preferably is a minimum 
threshold value for the baseline of the thorn, i.e. thorns occurring at very low amplitude 
are disregarded because they are considered to relate to background noise rather than to 
voice. 
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Fig. 2 is a graphic illustration of a voice segment 32, including a number 

of thorns 34. 

A "plateau" is a local flatness in the voice wave. For example, a plateau 
may be defined as a flat sequence whose length is more than a predetermined minimum 
threshold and is less than a predetermined maximum threshold. The maximum threshold 
is required to differentiate local flatness from a period of silence. A sequence may be 
regarded as flat if the difference in amplitude between consecutive samples is less than a 
predetermined threshold such as 5 amplitude units if 8-bit memory is employed. 

Fig. 3 is a graphic illustration of a voice segment 36, including a number 
of plateaus 38. In Appendix A, plateaus are termed "jumps". 

The system of the present invention typically operates in one of two 

modes: 

a. Calibration — building a profile of the subject's truth/neutral emotional 
state by monitoring a subject while the subject is not lying and/or is in a neutral 
emotional state. 

b. Testing - Comparing a subject's speech to the profile of the subject's 
truth/neutral emotional state as established during calibration, in order to establish 
emotional state and/or whether or not the subject is being truthful. 

If the system is to be used in calibration mode, the method proceeds from 
step 50 to step 60. If the system is to be used in testing mode, the method proceeds from 
step 50 to step 80. 

Step 60: If step 60 is reached, this indicates that the current segment has 
been processed for calibration purposes. Therefore, the thorn and plateau information 
derived in step 40 is stored in a calibration table. 

The processes of steps 20 - 50 are termed herein "voice recording 
entering processes". If there are more voice recordings to be entered for calibration 
purposes, the method returns to step 20. If entry of all voice recordings for calibration 
purposes has been completed (step 70), the method proceeds to step 80. 

Step 80: Build profile of truth/neutral emotional state for the subject who 
is currently being tested. This completes operation in calibration mode. Subsequently, the 
system enters testing mode in which the subject's voice recordings are compared to his 
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tnith/neutral emotional profile in order to identify instances of falsehood or heightened 
emotion. The subject's profile typically reflects central tendencies of the thorn/plateau 
information and is typically adjusted to take into account artifacts of the calibration 
situation. For example, due to natural stress at the beginning of the calibration process, 
the initial voice recordings may be less reliable than subsequent voice recordings. 
Preferably, to obtain a reliable indication of central tendencies, extreme entries in the 
calibration table may be discarded. 

Steps 90 onward pertain to the testing mode. 

Step 90: Compare thorn/plateau information of current segment to the 
truth/neutral emotion profile computed in step 80. 

Step 100; Threshold the results of the comparison process of step 90 in 
order to categorize the current segment as being indicative of various emotions and/or of 
falsehood. 

Step 105: Optionally, compensate for carryover. The term "carryover" 
refers to a residual emotional state carrying over from an "actual" emotional state 
occasioned by a first perceived situation, wherein the residual emotional state lingers 
after the first perceived situation has already terminated. An example of a suitable 
implementation for step 105 is described herein in the flowchart of Fig. 8. 

Step 110: Display a message indicating the category determined in step 

100. 

Step 120: If there are additional segments of voice to be analyzed, return 
to step 20. Otherwise, quit. Any suitable number m of segments may be used for 
calibration such as 5 segments. 

Fig. 4 is a simplified flowchart illustration of a preferred method for 
performing step 40 of Fig. IB. As described above, in step 40, thom/plateau information 
is generated for the crucial portion of a current voice recording segment. 

The current length of the plateau is termed "jj", 

"JjmapGO is the number of plateaus whose length is exactly jj. 

"Plat" is the counter counting the number of plateaus regardless of length. 

"Thorn" is the counter counting the number of thorns. 

n is the number of samples in a crucial portion under test. 
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In step 1 50, the thorn and plateau counters are reset. 

In step 160, start loop on all crucial portion samples. The loop is started 
at the first crucial sample and terminates at the last crucial sample minus 2. 

In step 164 the amplitudes of the samples in the loop are recorded. 

In steps 170 and 180 the thorns are detected, and in steps 190, 195, 200 
and 210 the plateaus are detected. 

In step 200, if the length of the candidate plateau is between reasonable 
bounds, such as between 3 and 20, increment the number of plateaus of length jj and 
increment Plat, the total number of plateaus. Otherwise, i.e. if the length of the candidate 
plateau is less than 3 or more than 20, the candidate plateau is not considered a plateau. 

Whether or not the candidate plateau is deemed a "real" plateau, the 
plateau length, jj, is zeroed (step 210). 

Step 220 is the end of the loop, i.e. the point at which all samples in the 
sequence have been checked. 

In step 230, compute the average (AVJ) and standard error (JQ) of the 
plateau length variable, jjmap. 

In step 240, compute SPT and SPJ. SPT is the average number of thorns 
per sample, preferably suitably normalized. SPJ is the average number of plateaus per 
sample, preferably suitably normalized. 

* * According to the illustrated embodiment, emotional status detection is 
multi-dimensional, i.e. emotional status is derived from the speech information via a 
plurality of preferably independent intermediate variables. 

Fig. 5 is a simplified flowchart illustration of a preferred method for 
implementing the truth/neutral emotion profile building step of Fig. IB. 

In Fig. 5, SPT(i) is the SPT value for segment i. 

MinSPT is the minimum SPT value measured in any of the m segments. 

MaxSPT is the maximum SPT value measured in any of the ra segments. 

MinSPJ is the minimum SPJ value measured in any of the m segments. 

MaxSPJ is the maximum SPJ value measured in any of the m segments. 

MinJQ is the minimum JQ value measured in any of the m segments. 

MaxJQ is the maximum JQ value measured in any of the m segments. 
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ResSPT is the size of the range of SPT values encountered during 
calibration. More generally, ResSPT may comprise any suitable indication of the extent 
of variation in the number of thorns which may be expected, when the subject is in a 
truth/neutral emotional state. Therefore, if the number of thorns in a speech segment is 
non-normative, with relation to ResSPT, then the subject can be said to be in a non- 
neutral emotional state such as an emotional state characterized by excitation or even 
arousal. ResSPT is, therefore, typically an input to the process of evaluation of SPT 
values generated during unknown emotional circumstances. 

ResSPJ is the size of the range of SPJ values encountered during 
calibration. More generally, ResSPJ may comprise any suitable indication of the extent of 
variation in the number of plateaus which may be expected, when the subject is in a 
truth/neutral emotional state. Therefore, if the number of plateaus in a speech segment is 
non-normative, with relation to ResSPJ, then the subject can be said to be in a non- 
neutral emotional state, such as an emotional state characterized by a feeling of internal 
contradiction or cognitive dissonance. ResSPJ is, therefore, typically an input to the 
process of evaluation of SPJ values generated during unknown emotional circumstances. 

ResJQ is the size of the range of JQ values encountered during calibration 
which serves as a baseline value for evaluation of JQ values generated during unknown 
emotional circumstances. 

It is appreciated that the baseline need not necessarily be a 4-dimensional 
baseline as shown in Fig. 5 but may alternatively be even one-dimensional or may have 
many more than 4 dimensions. 

Fig. 6 is a simplified flowchart illustration of a preferred method for 
performing step 90 of Fig. IB on a particular segment As described above, in step 90, 
thorn/plateau information of a current segment is compared to the truth/neutral emotion 
baseline computed in step 80. 

Step 400 is an initialization step. 

Step 410 computes the deviation of a current crucial portion from the 
subject's previously computed truth/neutral emotional state profile. In the illustrated 
embodiment, the deviation comprises a four-dimensional value including a first 
component related to the number of thorns, a second component related to the number 
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of plateaus, a third component related to the standard error in the plateau length and a 
fourth component related to average plateau length. However, it is appreciated that 
different components may be employed in different applications. For example, in some 
applications, the distribution of thorns (uniform, erratic, etc.) over a time interval may be 
useful in deriving information regarding the subject's emotional state. 

"Breakpointf" is a threshold value characterizing the acceptable range of 
ratios between average number of thorns in truth/neutral emotional circumstances, and 
the particular number of thorns in the current crucial portion. 

"Breakpoint^ is a threshold value characterizing the acceptable range of 

ratios between average number of plateaus in truth/neutral emotional circumstances, and 
the particular number of plateaus in the current crucial portion. 

"BreakpointQ" is a threshold value characterizing the acceptable range of 
ratios between average standard error of the number of plateaus in truth/neutral 
emotional circumstances, and the particular standard error in the number of plateaus in 
the current crucial portion. 

"Breakpoint ^" is a threshold value characterizing the acceptable range of 
ratios between average plateau length in truth/neutral emotional circumstances, and the 
particular average plateau length in the current crucial portion. 

Steps 420 - 470 update the subject's profile to take into account the new 
information garnered from the current segment. In the illustrated embodiment, only the 
ResSPT and ResSPJ values are updated, and only if the deviation of a current crucial 
portion from the subject's previously computed truth/neutral emotional state profile is 
either very large (e.g. exceeds predetermined ceiling values) or very small (e.g. fells 
below certain typically negative predetermined floor values). If the deviation of the 
current crucial portion from the truth/neutral profile is neither very large nor very small 
(e.g. frills between the ceiling and floor values), the subject's profile is typically left 
unaltered at this stage. 

In steps 460 and 470, if zzSPT and zzSPJ, respectively, are very close to 
zero, then the system's sensitivity is increased by decrementing ResSPT and ResSPJ 
respectively. 



12 



WO 99/31653 



PCT/IL98/006I3 



Step 480 generates suitable, typically application-specific combinations of 
the deviation components computed in step 410. These combinations are used as a basis 
for suitable emotional classification criteria, such as the emotional classification criteria 
specified in Fig. 7. The emotional classification criteria of Fig. 7 determine whether or 
not to classify the subject as exaggerating, as being untruthful, as being evasive, as being 
confused or unsure, as being excited, or as being sarcastic. However, it is appreciated 
that different emotional classifications may be employed in different situations. 

In the illustrated embodiment, the SPT information is mainly used to 
determine the excitement level. More specifically zzSPT is used to determine the value of 
crEXCITE, which may also depend on additional parameters as crSTRESS. For example 
a crEXCITE value of between 70 and 120 may be deemed normal, whereas values of 
between 120 and 160 may be deemed indicative of medium excitement and values 
exceeding 160 may be deemed indicative of high level excitement. 

In the illustrated embodiment, the SPJ information is mainly used to 
determine feelings of psychological dissonance. For example, a zzSPJ value of between 
0.6 and 1.2 may be deemed normal, whereas a value of between 1.2 and 1.7 may be 
deemed indicative of confusion or uncertainty. A value exceeding 1 .7 may be deemed 
indicative of awareness of voice on the part of the subject, and/or of an attempt of the 
subject to control his voice. 

In the illustrated embodiment, the zzJQ and crSTRESS values are mainly 
used to determine the stress level. For example, a crSTRESS value of between 70 and 
120 may be deemed normal, whereas values of over 120 may be deemed indicative of 
high stress. 

In the illustrated embodiment, the AVJ information is used to determine 
the amount of thought invested in spoken words or sentences. For example, if crTHINK 
exceeds a value of 100 then the amount of thought invested in a last sentence spoken is 
higher than the amount of thought invested in the calibration phase. This means that the 
person is thinking more about what he is saying than he did in the calibration phase. If 
the value is less than 100 the person is thinking less about what he is saying than he did 
in the calibration phase. 
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In the illustrated embodiment the crLIE parameter is used to determine 
truthfulness. A crLDE value to 50 may be deemed indicative of untruthfulness, values of 
between 50 and 60 may be deemed indicative of sarcasm or humor, values of between 60 
and 130 may be deemed indicative of truthfulness, values of between 130 and 170 may 
be deemed indicative of inaccuracy or exaggeration, and values exceeding 170 may be 
deemed indicative of untruthfulness. 

Referring back to Fig. 6, the parameters mentioned above may receive the 
following values: 

Breakpoint j = Breakpoint j = BreakpointQ = Breakpoint a = 1.1 

Ceilingy = Ceilingj =1.1 

Floorj = Floorj = -0.6. 

Increment^ = Incrementj = Decrementj = Decrementj = 0. 1 
Minimalj = Minimalj = 0.1 

It is appreciated that all of the numerical values are merely examples and 
are typically application-dependent. 

Fig. 7 illustrates the method for converting the various parameters in to 
messages which may be displayed, as shown for example in Fig. 1 . 

Fig. 8 represents a method for fine tuning the truth/neutral emotional 

state. 

Appendix A is a computer listing of a software implementation of a 
preferred embodiment of the invention shown and described herein which differs slightly 
from the embodiment shown and described herein with reference to the drawings. 

A suitable method for generating the software implementation is as 

follows: 

a. On a PC equipped with a microphone, a sound card and Visual Basic^M 

Version 5 software, generate a new project. 

The recording setting of the sound card may operate in accordance with 
the following parameters: 1 1 KHz, 8 bit, mono, PCM. 

t>. Place a timer object on the default form which appears in the new project. 

The timer object is called "timerr. 
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c. Place an MCI multimedia control object on the form. This object is called 
"mmcontrotl". 

d. Place 5 label objects on the form. These labels are called label 1, labe!2, 
labeB, Iabel4 and labe!6. 

e. Create 4 label arrays on the form. Rename the arrays as follows: 
SPT(0..4), SPJ(0..4), JQ (0..4), AVJ(0..4). 

f Place a command button on the form and change its caption property to 

end. The command button is called "command 1". 

g. Generate code for the form by keying in the pages of Appendix A which 
are headed "forml". 

h. Add a module to the project. Generate code for the module by keying in 
the pages of Appendix A which are headed ,, Feelings_detector , \ 

i. Connect a microphone to the PC. 

j. Press (F5) or "run" in order to start the application. 

Fig. 9 is a pictorial illustration of a screen display depicting the form, in 
design mode, just before starting the application. 

Fig. 10 is a pictorial illustration of a screen display depicting the form, in 
run mode, during calibration to a particular subject. 

Fig. 1 1 is a pictorial illustration of a screen display depicting the form, in 
run mode, during testing of a subject. 

The values of the CoR_msgX variable in Appendix A are as follows: 

I — truthfulness, 2 — sarcasm, 3 — excitement, 4 — 
confusion/uncertainty, 5 — high excitement, 6 — voice manipulation, 7 — lie/ false 
statement, 8 — exaggeration/inaccuracy. 

Variables carrying data of the current crucial portion have names which 
begin with the following characters: cor_. 

Baseline factors have names which begin with the following characters: 

cal_. 

Breakpoint factors have names which begin with the following characters: 
ResSPT and resSPJ are called ResT and ResJ respectively. 
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Fig. 12 is a simplified functional block diagram illustration of a system for 
detecting emotional states which is constructed and operative in accordance with a 
preferred embodiment of the present invention and which is operative to perform the 
method of Fig. IB. As shown, the system of Fig. 12 includes a voice input device such as 
a tape recorder 700, microphone 710 or telephone 720 which generates speech which is 
input by an emotion detection workstation 735 via an A/D converter 740. A voice 
window recorder 750 typically partitions the incoming speech-representing signals into 
voice windows or segments which are analyzed by a voice window analyzer 760. The 
voice window analyzer compares the voice windows or segments to calibration data 
stored in unit 770. The calibration data is typically derived individually for each 
individual subject, as described in detail above. A display unit or printer 780 is provided 
for displaying or printing an emotional status report, preferably on-line, for the user of 
the system. 

It is appreciated that the software components of the present invention 
may, if desired, be implemented in ROM (read-only memory) form. The software 
components may, generally, be implemented in hardware, if desired, using conventional 
techniques. 

It is appreciated that the particular embodiment described in the Appendix 
is intended only to provide an extremely detailed disclosure of the present invention and 
is not intended to be limiting. 

It is appreciated that various features of the invention which are, for 
clarity, described in the contexts of separate embodiments may also be provided in 
combination in a single embodiment. Conversely, various features of the invention which 
are, for brevity, described in the context of a single embodiment may also be provided 
separately or in any suitable subcombination. 

It will be appreciated by persons skilled in the art that the present 
invention is not limited to what has been particularly shown and described hereinabove. 
Rather, the scope of the present invention is defined only by the claims that follow: 
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The following code should be written on the form object: 
Fprpil 

Private Sub Command l_CIick() 
End 

End Sub 

Private Sub Form_Load() 

' Set properties needed by MCI to open. 

a = mciSendStringO'setaudio waveaudio algorithm pern brtspersample to 8 _ 
bytespersec to 1 1025 input volume to 100 source to avarage", 0, 0, 0) 

MMControll. Notify = False 
MMControll.Wait = True 
MMControll. Shareable = False 
MMControll. DeviceType — "WaveAudio" 
MMControll. filename = "C:\buf.WAV" 

1 Open the MCI WaveAudio device. 
MMControll. Command = "Open" 

' Define constants 
CR_BGlevel =15 ' Background level barrier 
CRJBGfilter = 3 1 Vocal wave smoother 
CRDATAstr = "" ' Reset Data String 
CR_mode = i 
CONS_S ARIOSO 

CONSJLIE11 = 130: CONS_LIE12 = 175 
CONS_LowzzT = -0.4: CONSJDghzzT = 0.3 
CONS_LowzzJ = 0.2: CONS^HighzzJ = 0.7 
CONS_RES_SPT = 2: CONS_RES_SPJ = 2 
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C0NS_BGfiler=3 

* Set timer object to work every 0.5 sec 
Timer 1. Interval = 500 
Timerl .Enabled = True 

1 set display 

Label 1. Caption = "System decision** 
Label2. Caption = "Global stress:" 
LabeD. Caption = "Excitement;" 
Label4 Caption = "Lie stress:" 
MMControll. Visible = False 

End Sub 

Private Sub Timerl_Timer() 
Static been 

On Error Resume Next 

MMControll. Command = "stop" 

MMControll. Command = "save* 

MMControll. Command = "close" 

* read data from file 

ff= MMControll .filename 

Dim kk As String * 6500 

kk = Space(6500) 

Open flf For Binary Access Read As #1 
Get #1, 50, kk 
Close #1 
Kiilff 

MMControll. Command = "open" 
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a = MMControII.ErrorMessage 

MMControll. Command = "record" 

CRJD AT Astr = kk 
IfOP_stat = 0Then 

OP_stat = 1 ' first round or after recalibration demand 

been = 0 

End If 
If been < 5 Then 

Label 1. Caption = "Calibrating./ 

Call Calibrate 1 Perform calibration 

1 get calibration status by CR_msgX 

If CoR msgX > -I Then 1 good sample 
been = been + 1 
End If 

Exit Sub 

Else 

OP_stat = 2 1 Checking status 
Call CHECK 

' get segment status by CRjnsgX 
End If 

If CoRjnsgX < 0 Then Exit Sub 1 not enogh good samples 

LabeM.Caption = "Lie stress.:" + Format(Int(CR_LrE)) 
Label2.Caption = "Global stress:" + Format(Int(CR_STRESS)) 
LabeB.Caption = "Excite Rate:" + Format(Int(CR_EXCITE)) 
Label6.Caption = "Thinking Rate:" + Format(Int(CR_THINK)) 

been = been + 1 
Select Case CoR_msgX 
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Case 0 

ans = "background noise" 
Case 1 

ans = "TRUE" 
Case 2 

ans = "Outsmart" 
Case 3 

ans = "Excitement" 
Case 4 

ans = "Uncertainty " 
Case 5 

ans = "High excitement" 
Case 6 

ans = "Voice manipulation / Avoidance / Emphasizing" 
Case 7 

ans = "LIE" 
Case 8 

ans = "Inaccuracy H 
End Select 
Label 1. Caption = ans 

End Sub 

Sub Calibrate() 

Call CUT_sec 
If CR_noSMP < 800 Then 
' no samples 

CoR_msgX = -l 

Exit Sub 

End If 



21 



WO 99/31653 



PO 1L98/00613 



1 Scan thorns 
CONS_RES_SPT - 2 
CONS_RES_SPJ « 2 

CaU scanTJ 

If Int(CoR_spT) = 0 Or Int(CoR_AVjump) = 0 Or Int(CoR_QJUMP) = 0 Or 
Int(CoR_SPJ) = 0 Then 

CoR_msgX = -l 

Exit Sub 

End If 

tot_T = 0: tot_J = 0: tot JQ = 0: totavj = 0 
minspT = 1000: minspJ = 1000: minJQ = 1000 
For a = 0 To 4 

If SPT(a).Caption = 0 And SPJ(a).Caption = 0 Then 

SPT(a).Caption = Int(CoR_spT) 

SPJ(a)Caption = Int(CoR_SPJ) 

JQ(a).Caption = Int(CoR_QJUMP) 

AVJ(a).Caption = Int(CoR_AVjump) 

Exit For 

End If 
tot_T = totJT + SPT(a) 
tot_J = tot_J + SPJ(a) 
tot_JQ = totJQ + JQ(a) 
tot_avj = tot_avj + AVJ(a) 

If Val(SPT(a).Caption) < minspT Then minspT = Val(SPT(a). Caption) 
If Val(SPT(a).Caption) > raaxspT Then maxspT = Val(SPT(a).Caption) 
If Val(SPJ(a).Caption) < minspJ Then minspJ = Val(SPJ(a).Caption) 
If Val(SPJ(a).Caption) > maxspj Then maxspJ = VaI(SPJ(a).Caption) 
If VaI(JQ(a).Caption) < minJQ Then minJQ = Val(JQ(a).Caption) 
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If VaI(JQ(a).Caption) > maxJQ Then maxJQ = Val(JQ(a). Caption) 
Next a 

'calc current CAL factors 
CAL_spT = (tot JT + Int(CoR_spT)) / (a + 1) 
CAL_spJ » (tot J + Int(CoR_SPJ)) / (a + 1) 
CALJQ = (totJQ + Int(CoRJJJUMP)) / (a + 1) 
CAL_AVJ = (tot_ayj + Int(CoR_AVjump)) / (a + 1) 

' calc resolution per factor 
On Error Resume Next 
Ifa> 1 Then 
res_T = maxspT / minspT 
res_J = maxspJ / minspJ 
End If 
CoR_msgX = 0 
End Sub 

Sub CHECKO 

CallCUT_sec 
If CR_noSMP < 800 Then 
' no samples 

CoR_msgX = -1 

Exit Sub 

End If 

CONS_RES_SPT = 2 
CONS_RES_SPJ = 2 

CaU scan TJ 
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If Int(CoR_spT) = 0 Or Int(CoR_AVjump) = 0 Or Int(CoR_QJUMP) = 0 Or 
Int(CoR_SPJ) = 0 Then 

CoRmsgX «= - 1 

Exit Sub 

End If 

Call analyze 
Call decision 

' Fine tune cal factors 

CAL spT = ((CAL_spT * 6) + CoR_spT) \ 7 
CAL spJ = ((CAL_spJ * 6) + CoR_SPJ) \ 7 
CALJQ = ((CALJQ * 9) + CoR_QJUMP) \ 10 
C AL_AVJ = ((C AL_AVJ * 9) + CoR_A Vjump) / 1 0 
End Sub 

The following code should be written in a new Module object: 
Feelings detector 

'Declaration section 

Global Fname -filename 

Global CR_BGfilter'-BackGround Filter 

Global CR_BGievel ' - BackGround level 

Global CR_D ATAstr 

Global CR_noSMP nomber of samples 

Global res_J, res_T 

Global CoR spT, CoR_SPJ, CoR_Ayjump, CoR_QJUMP 
Global CoR msgX, CR_retDATAstr 
Global SMP(IOOOO) As Integer 
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Global OP_stat 

• ** Calibration factors 

Global CAL spJ, CAL spT 

Global CAL_JQ, CAL_AVJ 

Global BP_J, BP_T '- CALIBRATION break points 

Global WI J, WI T, WI JQ Wigth of factors in calc. 

Global CR_zzT, CR zzJ 

Global CR_STRESS, CRJLIE, CR_EXCITE, CR_THINK 

Global CR_RESfilter ' -resolution filter 

' Constants for decision 

Global CONS_SARK 

Global CONS_LIE 1 1 , CONS_LEl 2 

Global CONS LowzzT, CONSJHighzzT 

Global CONSJLowzzJ, CONSJfighzzJ 

Global CONS_RES_SPT, CONS_RES_SPJ 

Declare Function mciSendString Lib "winmnvdlT Alias "mciSendStringA" (ByVal 
IpstrCommand As String, ByVal IpstrRetumString As String, ByVal uRetumLength As 
Long, ByVal hwndCallback As Long) As Long 

Sub analyze() 

On Error Resume Next 

CRJJOE = 0 

CR_STRESS = 0 

CREXCITE = 0 

If (CoR spT = 0 And CoR SPJ = 0) Or CR noSMP = 0 Then 
CR_rasg = "ERROR" 
Exit Sub 
End If 



25 



WO 99/31653 



PCT/IL98/00613 



If CoR_SPJ = 0 Then CoR_SPJ = 1 
If CoRjpT = 0 Then CoR_spT = 1 

On Error Resume Next 
rrJ = res_J: rrT = res_T 

BP_J = 1.1:BP_T«U 

zz_spj = (((C AL_spJ / Int(CoR_SPJ)) - BP_J) / rrJ) 

If zz_spj > -0.05 And zz_spj < 0.05 Then res_J = res_J - 0. 1 

If res J < 1 .3 Then res_J « 1 .3 

If zzspj < -0.6 Then 

zz_spj = -0.6 

res J = res_J + 0. 1 

End If 
Ifzz_spj> 1.2 Then 

zzspj = 1.2 

res_J = res_J + 0. 1 

End If 

If res_J > 3.3 Then res_J = 3.3 
CR_zzJ = zz_spj 

zz spT = (((CAL_spT / CoR_spT) - BPJT) / nT) 
CRzzT = zz_spT 

Ifzz spT > -0.05 And zz_spT < 0.05 Then res_T = res_T - 0. 1 
If res_T < 1.3 Then resJT = 1.3 
If zzspT < -0.6 Then 

zz_spT = -0.6 

res JT = res_T + 0. 1 

End If 
Ifzz_spT> 1.2 Then 



26 



V»> '9/3165J 



PCT7IL98/00613 



zz_spT = 1 .2 
res_T = res_T + 0. 1 
End If 

If res_T > 3.3 Then res_T = 3.3 

WI_J = 6: WI_T = 4 

CR_STRESS = Int((CoR_QJUMP / CALJQ) * 100) 
ggwi = WI_J * WIT 

CR_LIE = ((zz_spT + I) * WI_T) * ((zz_spj + 1) * WI_J) 

CR_LE = ((CR_LIE / ggwi)) * 100 

CRJLIE = CR_LIE + Int((CoR_QJUMP - CALJQ) * 1.5) 

CR_THINK = Int((CoR_AVjump / CAL AVJ) * 100) 

CR_EXCITE = ((((((CR_zzT) / 2) + 1) * 100) * 9) + CR_STRESS) / 10 

• ********* END OF Phase 2- ***"»* 

If CR_LE > 210 Then CR_LIE = 210 

If CR_EXCITE > 250 Then CR_EXCITE = 250 

If CR_STRESS > 300 Then CR_STRESS = 300 

If CR_LIE < 30 Then CR_LIE = 30 

If CR_EXCITE < 30 Then CR_EXCITE = 30 

If CR_STRESS < 30 Then CR_STRESS = 30 
End Sub 

Sub CUTsecO 

CR_noSMP - 0 

If CRDATAstr = ■" Then 
CR_msg = "ERROR! - No data provided" 
Exit Sub 
End If 

CR AUTOvol = 1 ' Auto amplifier 
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CoR_volume = 3 ' default 
CR_minSMP = 800 ' default 
free = FreeFile 

' Break CR_DATAstr to bytes 
LocA= 1: LocB = 1 
BGAmin = 0 
BGAmax = 0 
MAXvolume = 0 
TestP = 0 
BRJLOW=-128 
BR_high = -128 
ddd = -128 

ddd = Int(ddd * (CoRjvolume / 3)) 

ddd - (ddd \ CR_BGfilter) * CR_BGfilter 

IfCR_AUTOvol= 1 Then 

' apply auto volume detect 

MAXvolume = 0 

For a = 1 To Len(CRJDATAstr) 

ccc = Asc(Mid$(CR_DATAstr, a, 1)) 

ccc = ccc- 128 

ccc = (ccc \ CR_BGfiher) * CR_BGfilter 

If (ccc > CR BGlevel Or ccc < 0 - CR_BGlevd) And ccc > ddd Then 
If Abs(ccc) > MAXvolume Then MAXvolume = Abs(ccc) 
If StartPos = 0 Then StartPos = a 
OKsmp = OKsmp + 1 
End If 

If MAXvohime > 1 1 0 Then Exh For 
Next a 

If OKsmp < 10 Then 
CR_msg = "Not enough samples !" 
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CR_noSMP = 0 
Exit Sub 
End If 

CoR_volume = Int(360 / MAXvolume) 
If CoR_volume > 16 Then CoR_volume = 3 
End If 

On Error Resume Next 
drect = " H : DRJiag = 0 
MAXvolume = 0 
LocA=0 
Done = 0 

89 For a = StartPos To Len(CRJOATAstr) - 1 
ccc = Asc(Mid$(CR_DATAstr, a, 1)): ccd = Asc(Mid$(CR_DATAstr, a + 1, 1)) 
ccc = ccc - 128: ccd = ccd - 128 
ccc = Int(ccc * (CoR_volume / 3)) 
ccd = Int(ccd * (CoR_volume / 3)) 
ccc = (ccc \ CR_BGfilter) * CR_BGfilter 
ccd = (ccd \ CR_BGfilter) * CR_BGfilter 

If (ccc> CR_BGievel Or ccc < 0 - CRJBGlevel) And ccc > ddd Then 

If Abs(ccc) > MAXvolume Then MAXvolume = Abs(ccc) 

fl = fl+l 

End If 
Iffl>5Then 

SMP(LocA) = ccc 

If BRJiigh < ccc Then BRJiigh = ccc 
If BR_LOW > ccc Or BR LOW - -128 Then BR LOW = ccc 
If (SMP(LocA) > 0 - CRJBGIevel And SMP(LocA) < CR_BGIevel) Or 
SMP(LocA) = ddd Then 
blnk = blnk+l 
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Else 
blnk = 0 
End If 
If blnk> 1000 Then 
LocA = LocA - 700 
Done = 1 

If LocA > CR minSMP Then Exit For 

Done = 0 

LocA = 0 

fl = 2:blnk = 0 

BRLOW = -128: BR_high = -128 
End If 
LocA = LocA + 1 
End If 
Next a 
Err = 0 

CRnoSMP = LocA 

If CR_noSMP < CR_rainSMP Then 

CR_msg = "Not enough samples !" 

Exit Sub 

End If 

CR_msg = "Completed O.K." 
End Sub 

Sub decisionO 

If CR zzT = 0 And CR zzJ = 0 And (CL spJ o Int(CoR_SPJ)) Then 
CRjmsg = "ERROR! - Required parameters missing !" 
Exit Sub 
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End If 

If CR_STRESS = 0 Or CRLIE = 0 Or CR_EXCITE = 0 Then 
CR msg = "ERROR! - Required calculations missing ! " 
Exit Sub 
End If 

CRmsgCode = 0 
CoRmsgX = 0 
sark = 0 

If CR_LIE < 60 Then 
CoRmsgX = 2 
Exit Sub 
End If 

5555 If ((CR_zzJ + 1) * 100) < 65 Then 

If ((CR_zzJ + 1) * 100) < 50 Then sark = 1 
CR_zzJ = 0.1 
End If 

If ((CR_zzT + 1) * 100) < 65 Then 

If ((CR_zzT + 1) * 100) < CONS_SARK Then sark = sark + 1 
CR_zzT = 0. 1 
End If 

LD3_B0RD1 = CONS_LIEl 1: LIE BORD2 = CONSJJE12 

If CR_LIE < LIE BORD1 And CR STRESS < LEE BORD1 Then 

CRmsgCode = CRmsgCode + 1 

End If 

If CR LIE > LIE BORD1 And CR LIE < LDE BORD2 Then 
CoRmsgX ■= 8 
Exit Sub 
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End If 

If CR_LIE > LIE_BORD2 Then 
If CR msgCode < 128 Then CRjnsgCode = CR msgCode + 128 
End If 

If CRzzJ > CONSJLowzzJ Then 
If CR_zzJ > CONS_HighzzJ Then 
CR_msgCode = CR_msgCode + 64 
Else 

CRjnsgCode = CRmsgCode + 8 
End If 
End If 

If CR_EXCITE > LEEBORD1 Then 
IfCREXCITE > UEBORD2 Then 

If (CR msgCode And 32) = False Then CR_msgCode = CR_msgCode + 32 
Else 

If (CR msgCode And 4) = False Then CR msgCode = CR msgCode + 4 
End If 
End If 

If CR msgCode < 3 Then 
If sark = 2 Then 

CRmsgCode = -2 

CoRmsgX = 2 

Exit Sub 

End If 
If sark= 1 Then 

If(CR_zzT > CONSJLowzzT And CR_zzT < CONS HSghzzT) Then 
CRmsgCode = -1 
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CoR_msgX = 2 
Else 

If CR_zzT > CONS HighzzT Then CoR_msgX = 7 
End If 

If (CR_zzJ > CONS LowzzT And CR_zzJ < CONS_HighzzT) Then 
CR_msgCode = -l 
CoR msgX = 2 
Else 

If CR_zzJ > C0NS_Highz2T Then CoRjnsgX = 7 
End If 
Exit Sub 
End If 
CRmsgCode = I 
CoRmsgX = 1 
Exit Sub 
End If 

If CR_msgCode > 127 Then 

CoR_msgX = 7 

Exit Sub 

End If 
If CR msgCode > 67 Then 

CoR_msgX = 8 

Exit Sub 

End If 

If CR msgCode > 63 Then 

CoRmsgX = 6 

Exit Sub 

End If 
If CR_msgCode > 3 1 Then 

CoR_msgX = 5 
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Exit Sub 

End If 
If CRjnsgCode > 7 Then 

CoRjnsgX = 4 

Exit Sub 

End If 
If CRjnsgCode > 3 Then 

CoR msgX = 3 

Exit Sub 

End If 
CoRjnsgX = 1 
End Sub 

Sub scan_TJ() 
ReDimjjump(IOO) 
CRmsg = " 
TestP = CRnoSMP 
CR_spT = 0 
CR_SPJ = 0 
If TestP <= 0 Then 
CR msg = "No. of samples not transmitted !" 
Exit Sub 
End If 

CR_minJUMP = 3 1 default 
CR_maxJUMP = 20 ' default 

jump = 0 

thorns = 0 

BIGthoms = 0 

For a = 1 To CR_noSMP 

jjtl = SMP(a): jjt2 = SMP(a + 1); jjt3 = SMP(a + 2) 
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' scan thoms 

If (jjtl <jjt2 And jjt2 > jjt3) Then 

If jjtl > 15 And jjt2 > 15 And jjt3 > 15 Then thoms = thorns + 1 
End If 

If Qjtl > jjt2 And jjt2 < jjt3) Then 

If jjtl < -15 And jjt2 < -15 And jjt3 < -15 Then thoms = thorns + 1 
End If 

If (jjtl > jjt2 - 5) And (jjtl < jjt2 + 5) And Qjt3 >]jt2 - 5) And (jjt3 < jjt2 + 5) Then 
sss = sss + 1 
Else 

If sss >= CR_minJUMP And sss <= CRmaxJUMP Then 

jump = jump + 1 

jjump(sss) =jjump(sss)+ 1 

End If 
sss = 0 
End If 
Next a 

AVjump = 0 
JUMPtot = 0 
CRQJUMP = 0 

Fora=lTolOO 
JUMPtot = JUMPtot +jjump(a) 
AVjump » AVjump + Qjump(a) * a) 
Next a 

If JUMPtot > 0 Then crAVjump = AVjump / JUMPtot 
Fora=lTolOO 

If jjump(a) > 1 Then QJUMP = QJUMP + ((jjuinp(a) * Abs(cr_AVjump - a))) ' • 
jjump(a)) 
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Next a 

CoR_spT = (Int(((thorns) / CRjioSMP) * 1000) / CONS_RES_SPT) 
CoR_SPJ « (Int((Gump) / CR_noSMP) * 1000) / CONS_RES_SPJ) 
CoR_QJUMP = Sqr(QJUMP) 
CoR_AVjump = cr_AVjump 

CR msg = "Thorns & Jumps scan completed 0;K" 
End Sub 
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CLAIMS 

1 . Apparatus for detecting emotional status of an individual, the apparatus 
comprising: 

a voice analyzer operative to input a speech specimen generated by the 
individual and to derive therefrom intonation information; and 

an emotion reporter operative to generate an output indication of the 
individual's emotional status based on said intonation information. 

2. Apparatus according to claim I wherein said speech specimen is provided 
over the telephone to said voice analyzer. 

3. Apparatus according to claim 1 wherein said report on the individual's 
emotional status includes a lie detection report based on the individual's emotional status. 

4. Apparatus according to claims 1-3 wherein said intonation information 
comprises multidimensional intonation information. 

5. Apparatus according to claim 4 wherein said multidimensional 
information comprises at least 3-dimensional information. 

6. Apparatus according to claim 5 wherein said multidimensional 
information comprises at least 4-dimensional information. 

7. Apparatus according to claims 1 -3 , 5-6 wherein said intonation 
information includes information pertaining to thorns. 

8. Apparatus according to claim 7 wherein said information pertaining to 
thorns comprises the number of thorns in a predetermined time period. 
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9. Apparatus according to claim 8 wherein said information pertaining to 
thorns comprises the distribution of thorns over time. 

10. Apparatus according to claims 1-3, 5-6, 8-9 wherein said intonation 
information includes information pertaining to plateaus. 

11. Apparatus according to claim 10 wherein said information pertaining to 
plateaus comprises the number of plateaus in a predetermined time period. 

12. Apparatus according to claim 1 1 wherein said information pertaining to 
plateaus comprises information pertaining to length of plateaus. 

13. Apparatus according to claim 12 wherein said information pertaining to 
length of plateaus comprises an average plateau length for a predetermined time period. 

14. Apparatus according to claim 12 wherein said information pertaining to 
length of plateaus comprises the standard error of plateau length for a predetermined 
time period. 

15. A lie detection system comprising: 

a multidimensional voice analyzer operative to input a speech specimen 
generated by an individual and to quantify a plurality of characteristics of said speech 
specimen; and 

a credibility evaluator reporter operative to generate an output indication 
of the individual's credibility, including detection of lies, based on said plurality of 
quantified characteristics. 

16. A multi-dimensional lie detection method comprising: 

receiving a speech specimen generated by an individual and quantifying a 
plurality of characteristics of said speech specimen; and 
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generating an output indication of the individual's credibility, including 
detection of lies, based on said plurality of quantified characteristics. 

17. Apparatus according to claims 1-3, 5-6, 8-9, 11-15 wherein said speech 
specimen comprises a main speech wave having a period and wherein said voice 
analyzer is operative to analyze the speech specimen in order to determine rate of 
occurrence of plateaus, each plateau indicating that a local low-frequency wave is 
superimposed onto the main speech wave; and 

wherein the emotion reporter is operative to provide a suitable output 
indication based on the rate of occurrence of plateaus. 

18. A method for detecting emotional status and comprising: 
establishing a multi-dimensional characteristic range characterizing an 

individual's range of emotion when at rest by: 

monitoring the individual for a plurality of emotion-related 
parameters, over a first period during which the individual is in an emotionally neutral 
state; and 

defining the characteristic range as a function of the range of the 
emotion-related parameters during said first period; and 

monitoring the individual for said emotion-related parameters, over a 
second period during which it is desired to detect the individual's emotional status, 
thereby to obtain a measurement of said plurality of emotion-related parameters, and 
adjusting said measurement to take into account said range. 

19. A method for detecting emotional status of an individual, the method 
comprising: 

receiving a speech specimen generated by the individual and deriving 
therefrom intonation information; and 

generating an output indication of the individual's emotional status based 
on said intonation information. 



WO 99/316: 3 



PCT/IL98/00613 




WO 99/31653 



PCT/IL98/006H 




START 



2/12 

FIG. 1B 



Initialization-Constants 

declaration 

(e.g. INIT-CONSTS) 

*■ = 



10 



Sample incoming recording 0.5 
second long. 



20 



Detect louder samples, Define 
samples to be tested & Adjust 
volume and Background level 
(e.g. "CUT_SEC) 



30 
J 



Scan detected samples, 
Compute test parameters 
(e.g. " Scan_TJ" )-Fig. 4 



40 




Write results in 
calibration table. 



60 



Compute diversion from the 
Truth pattern- in all dimensions 
(e.g. Analyze" -Fig. 6) 



Analyze all results and decide 

out going message 

(e.g. "DECISION" -Fig. 7) 



100 



, 70 

Have all 
recordings for 
^Calibration been, 
^Processed?, 



NO 



Fine tune profile of truth/ 
neutral emotion state-(Fig. 8) 






M05 




Display out going decision 



YES 


80 


Build profile of 




truth/neutral emotion 




state-(Fig. 5) 






WO 99/31653 



PCT/IL98/00613 




FIG. 3 



WO 99/21653 



PCT/IL98/00613 



FIG. 4 



4/12 




Reset "Thorns" & "plateaus 
counters. 



I 



Start Loop(n) on segment 
samples, from first sample 
to last- 2 sample 



A= Amplitude of sample(n) 
B=Amplitude of sample(n+l) 
C=Amplitude of sample(n+2) 



I 



// A<B>C then 
Thorn=Thorn+ 1 



170 



150 



160 



164 



L 



195 



If A>B<C then 
Thom=Thom+ 1 



180 



YES 




// 2<jj<21 then Jjmap(jj)= 
Jjmap(j3)+ 1:PLAT=PLAT+ 1 



200 



Let jj=Q 



210 



End loop(n) 



I 



220 



Compute average length of 
plateaus(AVJ) & square root of 
plateau diversity ( JQ). 

T 



230 



Compute normalized averages 
of SPT, SPJ. e.g: 
SPT=((THORN/n) x 1000)/2 

SPJ=((PLAT/n) x 1000)/2 



240 




END 



WO 99/31653 



PCT/IL98/II0613 



5/12 



FIG. 5 



(^START^) 



Compute 4 dimensional baseline, e.g: 
CalSPT= average of m SPT values in m 
segments respectively. 
CalSPJ= average of m SPJ values in m 
segments respectively. — ^ 
CaUQ=average of m JQ values in m 300 
segments respectively. 
CalAVJ= average of m AVJ values in m 
segments respectively. 



Find minSPT, maxSPT as min. & max. 
values of SPT in m segments. 
Find minSPJ, maxSPJ as min. & max. 
values of SPJ in m segments. 
Find minJQ, maxJQ as min & max. 310 
values of JQ in m segments. 



Compute: 

ResSPT=(maxSPT/minSPT) 
ResSPJ=(maxSPJ/minSPJ) 

ResJQ=(maxJQ/minJQ) 




WO 99/31653 



I'CT/IL98/006B 



6/12 



(^START^) 



FIG. 6 



Reset (to zero) crLIE, crSTRESS, 
crEXCITE, crTHINK & crMSC 



400 



Compute deviation from profile 

of truth/neutral emotional state, e.g.: 

Z zSPT=(((calSPT/SPT)-BreakPointT)/ResSPT) 

zzSPJ=(((calSPJ/SPJ)-BreakPoinU)/ResSPJ) 

zzJQ=(((caUQ/JQ)-Breo±PointQ)/ResJQ) 

zzA VJ=((calA VJ/A VJ)-BreakPointA) 



410 



I 



// zzSPT>ceilingT then zzSPT=ceilingT: 
ResSPT=ResSPT+incrementT 



I 



420 



// zzSPJ>ceilingJ then zzSPJ=ceilingJ: 
ResSPJ=ResSPJ+incrementJ 



430 



// zzSPT<floorT then zzSPT=floorT: 
ResSPT=ResSPT+incrernentT 



I 



440 



// zzSPJ<floorJ then zzSPJ=floorJ: 
R esSPT=ResSPT+ incrementJ 



450 



// lzzSPTl<minimalT then 
ResSPT=ResSPT -decrementT 



460 



// IzzSPJfcminimaU then 
ResSPJ=ResSPJ-decrementJ 



470 



1 



Compute: 

crUE=((zzSPT+ 1 )*(zzSPJ+ 1))*1 00 
crLIE=crUE+((JQ-caUQ)*1.S) 
crSTRESS=int (zzJQ*100) 
crTHINK=int (zzAVJ*100) 

nrF.YCATE=int(((((zzSPT/2)+ 1 )*900)+ crSTRESS)/1 0) 



480 



WO 99/31653 



PCT/1L98/006H 



7/12 



FIG. 7 



START 




If crLIE < 50 then crLIE=190-crLIE 



500 



515 



// 130<crLIE<170 then 
crMSG= " Exaggeration 



517 



// crUE=>170 then 
crMSC="LIE" 




crMSG= Evasive 



525 



crMSG= " Confusion/ 
Not sure" 



535 



crMSC="High level of 
Excitement" 



545 



crMSG=" Medium 
Excitement'' 



555 



r565 



crMSG= " Sarcastic' 



crMSG=" Truth 




END 



WO 99/31653 



PCI7IL98/00613 



8/12 



FIG. 8 




Compute: 

CalSPT=((CalSPT * 6)+SPT)/7 






Compute: 

CalSPJ=((ColSPJ x 6)+SPJ)/7 






Compute: 
CoUQ=((CaUQ 


* 9)+JQ)/10 






Compute: 

CaJAVJ=((CoJAVJ « 9)+AVJ)/10 



600 



610 



620 



630 



WO 99/31653 



PCT/IL98/0(X 1-3 



CD 
6 




WO 99/31653 



PCT/IL98/00613 



10/12 



X 

O 



O 



"S 

"5 

o 



AVJ(a) 


CO 


o 


O 


Q- 

co 


to 


| SPT(a) 


ao 



m 


CO 


O 


o 


oo 


CM 


o 


o 






O 
CM 


o 


o 


a> 

CM 


CO 


O 


o 



o 



to 
to 
a> 



o 
O 



CD 

E 



CO 
CO 
O) 

to 



CO 
<L) 

o 



>> 

o 
c 
O 

cn 
cn 
a> 

±5 

cn 
a> 

□ 



o 
o 



WO 99/31653 



PCT/IL98/006I3 



11/12 



>< 

g 
T 



O 



o 

"o 

o 



AVJ(q) 




o 


OO 


SPJ(o) 


r- 


SPT(a) 


to 



to 



CD 



cn 



in 



CD 



CNI 



cn 



o 



cn 

rO 

CO 
CO 

a> 

"to 

~o 

^> 
o 



CM 

cn 

O 

or 



o 

X 



o 

O 



CO 
CO 

a> 
"lo 



cn 
a> 
o 

cn 
c 



> ». 

o 
c 
o 

CO 
CO 
<L> 
L- 

"to 
a> 

"o 
> 

□ 



O) 

o 
c 



WO 99/31653 



PCT/IL98/00613 



12/12 



FIG. 12 



710 



} 

MICROPHONE 



4- 



700 



TAPE RECORDER 



720 



TELEPHONE 



A/D 



740 



RECORDING OF 
VOICE WINDOWS 



-750 







VOICE WINDOW ^ 
ANALYSIS 




\ 


CALIBRATION DATA 
STORAGE 



760 



■770 



EMOTION DETECTION WORKSTATION 



DISPLAY UNIT/PRINTER FOR 
OUTPUTTING EMOTIONAL 
STATUS REPORT, E.G 
IN REAL TIME 



OTHER VOICE 
INPUT DEVICE 



730 



735 



780 



INTERNATIONAL SEARCH REPORT 



iBtmttiaBal applicabo* No. 
PCT/U.91AM613 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(6) <}10L SftO 

US CL :704O72 
Accotdmg to IntofMbaaal Patent Cbnificstioo (IPC) or to both 



national classification sad IPC 



FUCLDS SEARCHED 



US. 



I (classification system followed by 
7047272. 270. 271. 273. 274. 275; 600/300. 407 



symbols) 



DnoMtatoliua searched other than minimum documentation to the extent that such documents sre included is the fields 
eone 



bese consulted during the mtBroational search (nemo of dam 
I worfcheftca. APS, 1EL online 



base and. where practicable, search terms used) 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with mdicetion. where appropriate, of the rcftevaa 



t passages 



Relevant to claim No. 



Y 
Y 
Y 



US 3,855,418 A (FULLER) 17 December 1974, Col.2, lines 4-68, 
Col.3, line 1- Col. 4, line 4. 

US 3,971,034 A (BELL, Jr. et al.) 20 July 1976, abstract, Col.2, 
line 44 - Col.8, line 53. 

US 4,093,821 A (WILLIAMSON) 06 June 1978, abstract 

US 5,029,214 A (HOLLANDER) 02 July 1991, abstract. 

US 5,148,483 A (SILVERMAN) 15 September 1992, abstract. 



1-19 

1-19 

1-19 

1 

I 



listed in the continuation of Box C. Q 



See palest family snncx. 



■ «r«ei 



t»a*ef I 
•arfar*. 



• «f«w«n 



«r SWary — iImIjmbj km mmm 



«UiaW4 oa or *IW a» i 



I Hang 4*m 



L* * «BMh mmf mrmrn 4a+ m <m (man? cUiM*) or wfckfc n 

M(aeidfiM) *V 
t u seihi t to «a «nl diackaan. om. ukutHucn or o«W 

Jfftns ambutsurtMUi 



l«0urvot»«a*n 



lprwr«0(j 



bans «bvw«t la • Mno« akikd ■ Om art 
du i— mmmbm *f cm mm pmK (amity 



Date of the actual completion of the international search 
22 MARCH 1999 


Dale of mailing of the international search report 

20 APR 1999 


Ness* end masting eddies, of the ISA/US 
CeejSBjsBoeer of Patents and Tisdemsrks 
Box PCT 

WaskiaftoaaC 20Z31 
FscshoJe No. (703) 305*3230 


Authorized officer 

VUAY CHAWAN ^ ^jjj 

Teleobonc No. f703) 305-3VW) 



INTERNATIONAL SEARCH REPORT 



Intora*tiofl&J application No. 
PCT/IL9oV006I3 



C (CoatuoAtioaX DOCUMENTS CONSIDERED TO BE RELEVANT 



Caftegoty* 



Citation of docs meat, with indication, where appropriate, of the relevant passages 



Relevant Co claim No. 



Y.E 



US 5,875,427 A (YAMAZAKI) 23 February 1999. abstract 



Fans PCT71SA/210 (coatbaatioa of second iheetXJuly 1992)* 



